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ABSTRACT 

This  paper  discusses  the  impact  of  display  pixel  shape,  size  and  spacing  on  displayed  image  quality.  The 
application  of  Fourier  Transforms  to  sampled  imagery  is  described.  A  characterization  function  for 
sampled  imagers  is  derived  by  examining  the  image  formation  of  a  point  source.  Misapplication  of  the 
Sampling  Theorem  is  also  discussed. 
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1.  INTRODUCTION 

Many  imaging  systems  have  a  modulation  transfer  function  (MTF)  such  that  the  product  of  the  transfer 
function  times  the  Fourier  Transform  of  the  scene  being  imaged  gives  the  Fourier  Transform  of  the  image 
being  displayed.  The  transfer  function  quantifies  how  well  the  imaging  system  passes  the  spatial 
frequency  content  of  the  scene.  We  often  judge  the  quality  of  an  imaging  system  by  looking  at  its  MTF. 
Further,  MTF  is  used  as  the  independent  variable  in  image  quality  and  task  metrics. 

An  MTF  cannot  be  defined  for  imaging  sensors  which  sample  the  scene.  However,  a  response  function 
can  be  defined  which  characterizes  the  performance  of  a  sampled  system  in  much  the  same  way  that  MTF 
characterizes  a  non-sampled  system.  The  response  function  consists  of  a  transfer  function  term  and  a 
spurious  response  term.  Both  terms  are  important  to  the  quality  of  sampled  imagery  and  should  be 
considered  independently  when  establishing  quality  metrics. 

This  paper  reviews  the  application  of  Fourier  Transform  theory  to  sampled  systems.  The  transfer  and 
spurious  response  terms  for  a  sampled  imager  are  derived.  Misapplication  of  the  Sampling  Theorem  is 
also  discussed. 


2.  CHARACTERIZING  A  SAMPLED  IMAGER 
2.1  Fourier  Transform  of  a  sampled  image 

In  a  sampled  imager,  the  scene  content  is  sampled  after  filtering  by  the  pre-sampling  MTF  of  the  sensor.  If 
the  sensor  samples  at  spatial  interval  (X),  then  the  Fourier  Transform  of  the  displayed  image  is:'’' 

Img(w)  =  SflnX)  Pix(w)  exp jnwX  (1) 


v  Sensor  horizontal  and  vertical  response  are  assumed  to  be  independent;  the  examples  in  this  paper  are  one  dimensional. 
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where:  f(x )  =  sampled  waveform 

=  h(x)  convolved  with  scn(x) 
h(x)  =  pre-sample  sensor  point  spread  function 
scn(x)  =  scene  intensity  distribution 
w  =  2  7i  times  spatial  frequency 
H(w)  =  Fourier  Transform  of  h(x) 

=  pre-sampled  MTF  of  sensor 
Img(w)  =  Fourier  Transform  of  image 
Pix(w)  =  Fourier  Transform  of  pixel  shape 

The  exponential  places  each  display  pixel  at  a  sample  position.  The  transform  of  the  image  is  simply  the 
sum  of  the  transforms  of  the  appropriately  placed  pixels. 

Equation  (1)  allows  the  calculation  of  the  image  transform  given  the  sample  values  and  the  pixel  MTF. 
However,  Equation  (1)  does  not  provide  insight  into  the  relationship  between  pre-sample  MTF,  sample 
spacing,  and  the  display  pixel  MTF. 

The  following  derivation  follows  a  method  often  used  to  prove  the  Sampling  Theorem.  1  >2  The  image  can 
be  constructed  by  convolving  the  pixel  shape  with  delta  functions  at  the  sample  points.  The  origin  of  the 
scene  coordinate  is  not  necessarily  a  sample  point,  so  an  offset  (d)  must  be  included  for  generality. 

img(x)  =  2  J  f(x'-d)  5(x'-nX)  pix(x'-x)  dx' 

&  n  -oo 

=  2  {f(x-d)  5(x-nX)}  *  pix(x) 
n 

where  (*)  means  convolution  and: 

Pix(x)  =  pixel  shape 

=  Fourier  Transform  of  Pjx(w) 
img(x)  =  image  intensity  distribution 
=  Fourier  Transform  of  Img(w) 

The  Fourier  Transform  of  f(x)  calculated  in  Equation  (1)  is  for  f(x)  windowed  to  the  sampled  interval.  It 
does  not  matter  in  Equation  (1)  whether  f(x)  is  actually  limited  to  the  sample  interval  or  not.  This  is 
important  in  the  new  derivation  because  we  need  to  define  what  we  mean  by  F(w). 

To  accommodate  the  following  derivation,  we  want  to  treat  f(x)  as  zero  outside  the  sample  interval.  That 
is,  the  Fourier  Transform  of  f(x)  is  the  transform  of  the  windowed  function.  This  allows  the  substitution  of 
an  infinite  set  of  samples  in  lieu  of  the  finite  set  of  samples;  we  now  sample  all  space.  Since  convolution 
in  space  results  in  multiplication  in  frequency: 

Img(w)  =_£  [F(w)  exp  ^Wd  *  6(w-nu)]  Pix(w) 

Img(w)  = J ^(w-nu)  exp j(w  nu)d  Pix(w)  (2) 

where:  u  =  2  n  /  X  (the  sample  frequency  in  radians) 


Figure  (1)  is  a  notional  plot  of  Img(w).  The  image  spectrum  can  be  represented  as  F(w)  replicated  at 
multiples  of  (u)  and  then  weighted  by  Pix(w).  Note  that  the  phase  of  the  replicas  varies  due  to  the 
e"j(w-mi)d  multiplier.  The  image  will  vary  with  sample  phase. 

If  Equation  (2)  is  applied  to  a  real  sensor,  then  scn(x)  is  the  scene  limited  by  the  sensor  field  of  view. 
Also,  h(x),  the  point  spread  function,  must  be  spatially  limited.  It  is  only  under  these  circumstances  that 
Equation  (2)  can  substitute  for  Equation  (1).  However,  a  spatially  limited  function  cannot  be  band 
limited.^  Aliasing  is  inevitable  in  a  practical  system.  The  replicas  of  F(w)  will  overlap,  preventing  an 
exact  reconstruction  of  the  sampled  signal.  Of  course,  the  effect  of  the  aliasing  can  be  very  minor. 

2.2  Response  function  for  a  sampled  imager 

In  order  to  establish  a  characterizing  response  function  RSp(w)  for  sampled  imagers,  we  examine  the 
image  formation  of  a  point  source. ^4  The  function  being  sampled  is  the  point  spread  function  of  the 
sensor,  and  the  Fourier  Transform  of  the  point  spread  function  is  the  sensor  pre-sample  MTF. 

Rsp(w)  =  _E  H(w-nu)  exp  nU')d  Pix(w) 

RSp(w)  is  symmetrical  about  zero  frequency.  Also,  in  practical  systems,  the  display  MTF  will  limit 
RSp(w)  to  the  first  two  terms. 

RSp(w)  =  H(w)  Pix(w)  +  H(w-u)  exp }Ud  Pix(w)  (3) 

The  first  term  is  exactly  analogous  to  MTF  for  an  analog  system;  it  represents  the  transfer  of  scene 
information  to  the  image.  The  second  term  represents  spurious  response  introduced  by  the  sampling  and 
reconstruction.  The  phase  factor  common  to  both  terms  was  dropped. 

2.3  Examples 

In  order  for  the  illustrations  to  be  clear  even  after  printing  and  reproduction,  large  pixels  and  gross  sample 
spacing  will  be  used  in  the  examples.  All  of  the  examples  are  based  on  the  picture  of  a  wall  clock  shown 
in  Figure  (2).  The  picture  was  taken  with  a  television  camera.  The  horizontal  and  vertical  MTF  were 
measured  and  are  shown  in  Figure  (3). 

In  all  of  the  following  examples,  the  same  display  pixel  size  and  shape  are  used  but  the  sample  spacing  is 
changed.  The  square  pixel  is  shown  in  the  upper  right  comer  of  each  picture.  The  transfer  and  spurious 
response  calculations  are  done  assuming  a  square  pixel  convolved  with  a  narrow  Gaussian  to  round  the 
edges.  The  Gaussian  dropped  to  5%  at  an  eighth  the  pixel  width. 

In  Figure  (4),  the  clock  image  is  down-sampled  by  two  in  each  direction.  There  are  four  sample  positions 
for  each  pixel  spacing  in  each  direction  resulting  in  sixteen  samples  per  pixel  area.  The  picture  is  well 
sampled,  and  the  large  pixels  are  well  overlapped.  The  image  is  blurry  but  sampling  artifacts  are  not 
apparent. 

Figure  (5)  shows  the  calculated  transfer  and  spurious  responses  for  the  image  in  Figure  (4).  Note  that 
spatial  frequency  is  in  object  space.  Transfer  response  cuts  off  just  above  0.2  cycles  per  milliradian  and 
goes  negative  beyond  that  frequency.  This  result  is  expected;  the  pixel  is  square  with  a  size  (in  object 
space)  of  4.8  milliradians.  Spurious  response  is  almost  zero  everywhere.  The  spurious  response  is 
centered  at  the  sample  frequency  of  0.83  cycles  per  milliradian  and  is  therefore  well  filtered  by  the  MTF 
of  the  large  pixel. 


In  Figure  (6),  the  clock  image  is  down-sampled  by  four  in  each  direction.  There  are  now  two  horizontal 
sample  positions  and  two  vertical  sample  positions  for  each  pixel.  The  image  is  clearly  interpretable,  but 
sampling  artifacts  have  appeared.  Some  edges  are  jagged,  pixel  intensities  on  the  numbers  are  incorrect, 
and  some  of  the  minute  markings  are  missing  or  out  of  place. 

Figure  (7)  shows  the  calculated  response  for  the  sample  spacing  used  in  Figure  (6).  The  transfer  response 
has  not  changed.  Now,  however,  the  sampling  frequency  has  been  cut  in  half  and  the  spurious  response  is 
centered  at  0.42  cycles  per  milliradian.  In  this  case,  the  display  MTF  cannot  filter  out  the  sampling 
artifacts. 

The  spurious  response  is  in  two  parts.  The  response  does  not  go  to  zero  at  zero  frequency.  The  spurious 
response  "reflects"  off  the  amplitude  axis;  the  phase  changes  because  the  imaginary  component  changes 
sign  after  reflection. 

Figure  (8)  shows  the  clock  image  down-sampled  eight  times  in  both  directions.  Pixel  size  is  the  same  as 
sample  spacing.  The  sampling  artifacts  are  quite  bad.  The  seven,  eight  and  nine  numerals  are  illegible,  and 
many  of  the  minute  markings  are  missing  completely.  Edges  are  very  jagged.  The  image  appears  to  be 
sharper  because  the  pixels  do  not  overlap. 

Figure  (9)  shows  the  calculated  transfer  and  spurious  responses  for  the  Figure  (8)  image.  Again,  transfer 
response  is  the  same  as  in  previous  examples.  Now,  however,  the  spurious  response  is  about  the  same  as 
the  transfer  response. 

The  final  example  represents  a  practical  problem  which  sometimes  occurs  in  modem  systems.  The  sensor 
is  well  sampled.  However,  the  image  is  presented  on  a  low  resolution  flat  panel  display  or  on  a  computer 
display  using  pixel  replication  to  make  the  image  larger.  In  this  case,  there  is  little  aliasing,  but  the 
spurious  response  can  still  be  very  annoying. 

Figure  (10)  shows  an  image  of  the  numeral  "3."  Pixel  size  at  the  display  is  the  same  as  in  the  previous 
examples.  The  image  is  magnified  eight  times  because  there  is  now  one  pixel  for  each  of  the  original 
samples. 

The  transfer  response  is  much  better  than  in  previous  examples;  see  Figure  (11).  The  pixel  size  in  object 
space  is  only  0.6  milliradians.  The  small  sample  spacing  (high  sample  rate)  moves  the  spurious  response 
away  from  the  transfer  response;  they  do  not  overlap.  However,  the  high  pixel  MTF  results  in  a  visible 
spurious  response.  In  this  case,  the  spurious  response  can,  of  course,  be  removed  by  proper  display 
techniques.^ 


3.  MISAPPLICATION  OF  THE  SAMPLING  THEOREM 

The  Sampling  Theorem  is  often  used  to  provide  a  "yes/no"  answer  to  the  complicated  question  of  whether 
a  sensor  and  associated  display  are  adequately  sampled.  The  Sampling  Theorem  is  actually  of  little  value 
in  evaluating  real  sensor  and  display  systems. 

The  Sampling  Theorem  does  not  quantify  either  the  penalty  of  under-sampling  or  the  benefits  of  one 
display  choice  over  another.  Further,  the  often  quoted  "two  samples  per  cycle"  criteria  represents  a 
mathematical,  idealized  extreme;  this  criteria  is  not  useful  with  real  sensors  and  displays. 

Nothing  in  this  discussion  should  be  construed  as  suggesting  that  sampling  at  a  rate  double  the  effective 
MTF  cutoff  is  inadequate.  For  most  sensors,  sampling  at  a  frequency  double  the  point  where  MTF 
reaches  ten  to  forty  percent  is  generally  adequate  for  decent  image  quality. 


3.1  Description  of  the  Sampling  Theorem 


The  Sampling  Theorem  states  that,  for  a  signal  f(x)  for  which  the  Fourier  Transform  has  no  components 
above  frequency  ^ fs^  inclusive,  the  function  can  be  entirely  reconstructed  by  the  series: 

00 

f(x)  =n£^n/fsamp)  sin(7ifsampx-n7r)/(7rfsampx-n7T) 

A  band-limited  function  can  be  uniquely  determined  from  its  values  at  a  sequence  of  equidistant  points, 
1/fsamp  aPaft.  The  above  series  is  used  to  reconstruct  the  function.  Each  term  in  the  above  series  is  a 
sample  function  [sin(x)/(x)],  also  referred  to  as  a  sine  wave.  The  sine  wave  amplitude  is  equal  to  the 
sample  value,  and  the  period  is  such  that  the  sine  wave  crosses  zero  at  all  other  sample  points.  The 
function  is  sampled  over  all  space,  and  each  sine  wave  extends  in  both  directions  over  all  space. 

3.2  Example 

Figure  (12)  shows  ten  cycles  of  a  sine  wave;  Figure  (13)  is  the  Fourier  Transform.  Taking  the  first  zero  as 
the  "band-limit,"  the  function  is  sampled  2.2  times  per  cycle.  The  asterisks  in  Figure  (12)  show  the  sample 
points. 

Figure  (14)  shows  one  of  the  sine  waves  generated  to  reconstruct  the  sampled  function.  Figure  (15)  shows 
the  result  of  sampling  and  reconstructing  one  period  near  the  middle.  Finally,  sine  waves  are  generated  for 
all  22  samples  and  added  together.  The  resulting  reconstruction  is  shown  in  Figure  (16). 

The  sixth  cycle  is  poorly  sampled  due  to  the  sample  phase.  It  reproduces  well  in  the  final  result  because  it 
is  surrounded  by  other  sampled  periods  of  the  sine  wave.  The  edges  are  poorly  sampled  also  but,  having 
neighbors  on  only  one  side,  they  do  not  reconstruct  to  the  full  amplitude. 

Note  that  the  Sampling  Theorem  does  not  suggest  that  a  single  cycle  of  a  sine  wave  can  be  reconstructed 
by  taking  two  samples.  The  Fourier  Transform  of  a  single  cycle  of  the  sine  wave  is  shown  in  Figure  (17). 
Using  the  first  zero  as  the  "band-limit,"  the  sample  rate  would  be  4.0  per  cycle.  If  the  sine  wave  were 
infinitely  extended,  then  the  Fourier  Transform  would  be  a  delta  function  at  the  sine  wave  frequency,  and 
just  over  two  samples  per  cycle  would  be  adequate. 

3.3  Discussion 

Complete  knowledge  of  a  band-limited  function  over  a  local  interval  defines  the  function  over  all  space.  1 
The  converse  is  also  true:  the  local  interval  can  be  constructed  by  sampling  at  remote  locations.  The 
Sampling  Theorem  uses  this  property  of  a  band-limited  function.  When  using  2.2  samples  per  cycle  in  the 
above  example,  a  single  cycle  of  the  sine  wave  was  not  reconstructed  by  sampling  only  that  cycle.  The 
middle  of  the  function  was  reconstructed  by  sampling  the  whole  function. 

Information  about  a  local  area  of  an  image  does  not  reside  at  remote  locations  in  the  image.  We  try  to 
engineer  the  point  spread  function  of  a  sensor  to  be  finite,  hopefully  rather  small.  Also,  spatially  discrete 
pixels  are  used  on  the  display.  The  tenets  of  the  Sampling  Theorem  do  not  apply. 

Further,  even  if  the  conditions  specified  in  the  Sampling  Theorem  were  constructed,  and  certainly 
reasonable  approximations  are  achievable,  the  Sampling  Theorem  does  not  suggest  that  a  single  cycle  of  a 
sine  wave  can  be  reconstructed  by  taking  two  samples.  The  "two  samples  per  cycle"  criteria,  so  often 
applied  when  evaluating  real  sensors,  is  a  misapplication  of  the  Sampling  Theorem. 


4.  CONCLUSIONS 


Fourier  analysis  does  apply  to  sampled  imagers  and  can  be  used  to  find  a  response  function  analogous  to 
the  MTF  of  analog  systems.  The  characterizing  function  for  a  sampled  imager  is  found  by  examining  the 
image  formation  of  a  point  source.  Sampled  systems  have  a  transfer  response  identical  to  the  MTF  of 
analog  systems.  However,  the  spurious  response  resulting  from  the  combination  of  sample  spacing,  pre¬ 
sample  MTF  and  display  pixel  shape  is  also  important.  With  sampled  imagers,  image  quality  and  task 
metrics  become  a  multivariate  problem,  with  the  transfer  and  spurious  response  of  the  imager  treated 
independently. 

The  "two  samples  per  cycle"  criteria  based  on  the  Sampling  Theorem  represents  an  idealized  limit.  The 
"two  sample"  criteria  does  not  apply  to  real  sensors  and  displays.  Further,  the  Sampling  Theorem  cannot 
be  used  to  quantify  either  the  penalty  of  under-sampling  or  the  benefits  of  one  display  choice  over 
another. 
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Figure  1.  Notional  plot  of  sampled  spectrum,  display  MTF,  and  resulting  image. 


Figure  2.  Picture  of  wall  clock. 


Figure  4.  Clock  down-  sampled  by 
two  in  each  direction.  Four  samples 
per  display  pixel  each  direction. 


Figure  6.  Clock  down-sampled  by 
four  in  each  direction.  Two  samples 
per  display  pixel  in  each  direction. 


Figure  5a.  Horizontal  transfer  and  spurious  response  for  sampling  in  Figure  4. 


Figure  5b.  Vertical  transfer  and  spurious  response  for  sampling  in  Figure  4. 


Figure  7a.  Horizontal  transfer  and  spurious  response 
for  sample  spacing  of  Figure  6. 
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Figure  7b.  Vertical  transfer  and  spurious  response  for  sampling  in  Figure  6. 
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Figure  9a.  Horizontal  transfer  and  spurious  response  for  sampling 
shown  in  Figure  8. 


Figure  9b.  Vertical  transfer  and 
spurious  response  for  sampling 
shown  in  Figure  8. 
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Figure  11a.  Horizontal  transfer  and  spurious  response  for  Figure  10. 
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Figure  lib.  Vertical  transfer  and  spurious  response  for  Figure  10. 
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Figure  12.  Ten  periods  of  a  sine  wave  sampled  2.2  times  per  cycle  at  the  (*). 


I 


I 


15 


2 


I 


J 15 


Sfcetial  ReqLercy 


Figure  13.  Fourier  Transform  of  sine  waves  in  Figure  12. 
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Figure  14.  One  sine  wave  of  the  Sampling 
Theorem  reconstruction. 


samples  of  the  6th  cycle. 


Figure  16.  Reconstruction  using  all  22  samples 
of  the  ten  cycles. 


